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ABSTRACT 


A real-time, single-camera, stereoscopic video imaging device for use with a 
standard 60 Hz camera and monitor is being developed. The device uses a single objective 
lens to focus disparate views of the object at the focal plane of the lens. Each view is 
represented by a set of parallel rays emanating from the object at a specific angle. The 
lens focuses these parallel rays to a single point at the focal plane. These views are then 
shuttered at the focal plane using a Liquid-Crystal Device (LCD) shutter such that one 
view at a time is passed to the camera. The camera then transmits alternating video fields 
(individual TV images) to the monitor, such that alternate fields display 
stereoscopically-related views of the object being imaged. The user views the monitor 
using off-the-shelf LCD stereoglasses, modified to allow synchronization with the 
standard field rate of the camera and monitor. The glasses shutter the light alternately to 
each eye so that the left eye views the left-hand image and the right eye views the right- 
hand image. The resulting 3-D image is independent of the user’s viewing angle or 


distance from the monitor. 
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I. INTRODUCTION 


The advent of modern video equipment has allowed real-time, remote imagery in 
environments as distant and extreme as outer space and the Moon. These videos have 
necessarily been two-dimensional (2-D) because of the nature of the camera system and 
the monitors on which the image is displayed. This provides adequate visual acuity for en- 
tertainment and most remote information gathering applications; however, many tasks are 
much more difficult or nearly impossible to perform with only 2-D visual cues. For these 
tasks, stereoscopic imagery provides the answer. 

Stereoscopic, or three-dimensional (3-D), imagery has been shown to provide sig- 
nificant improvements in time-to-task-completion, visual perception of the remote scene, 
and object detection and recognition during delicate teleoperation tasks using a remote 
manipulator [Ref. 1]. In addition, some remote teleoperation tasks have been found to be 
impossible without 3-D video feedback [Refs. 2,3]. 

Recent advances in Liquid-Crystal Device (LCD) shutter technology have pro- 
vided the impetus for a proliferation of stereoscopic devices that operate with two lenses 
and two cameras [Refs. 4,5,6,7]. Such devices can be time-multiplexed at twice the Na- 
tional Television System Committee (NTSC) standard field rate to provide alternating 
video fields (TV screen images) to a specially designed monitor in a process known as ‘al- 
ternating field technology’. The 3-D image is perceived when the monitor is viewed with 
LCD stereoglasses. 

Although this system has been used in numerous applications with great opera- 
tional success, it does have significant limitations. The two-camera system requires a 
larger communications bandwidth than the standard single NTSC video signal, limiting its 
application in extreme communications environments such as outer space, where increases — 
in signal bandwidth are expensive and technically challenging to achieve. Also, either a 
special monitor and special cameras that operate at twice the NTSC video field rate, or a 
complex hardware and software unit must be used that stores, processes, and time- 
multiplexes the two images [Ref. 5]. Finally, in many remote working environments, 


there is simply not enough working space to effectively employ an instrument that requires 











two cameras and two systems of lenses. For these reasons, the two-lens, two-camera 
stereoscopic video system has not found a strong foothold in many remote teleoperation 


fields. 


A. THESIS OBJECTIVES 


This thesis presents the preliminary design of a real-time, single-camera, stereo- 
scopic video system for use with NTSC standard cameras and video monitors. The objec- 
tive of this research is to produce a viable alternative to two-camera stereoscopic devices 
currently available. The single-camera Stereoscopic Video Device (SVD) is intended for 
specific 3-D video applications not met by the current technology. 

I. Description of a Real-time, Single-camera, Stereoscopic Video Device 

A real-time, single-camera, SVD has been developed for use with an NTSC stan- 
dard camera and monitor. A schematic of the device is shown in Figure 1-1. An in-depth 
discussion of the theory of operation of the device is presented in Chapter III. 

The system uses a state-of-the-art LCD shutter and a single camera to sequentially 
present alternating video fields (individual TV images) to the monitor. An image from the 
left-hand view of the object is displayed for one field, and the stereoscopically related 
right-hand view is displayed for the next field. The image fields are alternated at 60 Hz, 
which is the NTSC standard field rate for video cameras and monitors. The user views the 
monitor using off-the-shelf LCD stereoglasses, modified to allow synchronization with the 
standard field rate of the camera and monitor. The glasses shutter so that the left eye sees 
only the corresponding left-hand image and the right only the right-hand image. Because 
of the high field rate of the system, the brain does not interpret the inputs from the two 
eyes as separate images, rather, it fuses the two views into a single 3-D image in a process 
known as stereopsis (from the Greek words stereos meaning 'solid', and opsis meaning 'vi- 
sion’). The resulting 3-D image is independent of the user's viewing angle or distance from 
the monitor. 

The single-camera SVD is particularly well suited for near-field, precise, remote 


manipulations which use a single video camera for visual feedback. The entire imaging 








device can be packaged in a cylindrical unit with a diameter under 6 mm, providing 3-D 
visual accessibility for medical optical devices, space applications, and remote undersea 
operations. In addition, the use of a single camera detector as the video acquisition device 
permits 3-D imagery under extremely remote conditions in which communications band- 


width is limited (such as occurs in space communications). 
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Figure 1-1. Schematic of a Single-camera Stereoscopic Video Device. 

This schematic is an overhead view of the lens and shutter system for an SVD. 
Parallel rays emanating from the object at different angles with respect to the lens 
system centerline correspond to different views of the object. The disparate views 
of the object are focused by the objective lens to different points on the focal 
plane. The shutter passes only one set of views at a time to the camera detector. 


Other advantages of the single-camera stereoscopic video system include the fol- 
lowing: (1) the system works at the standard NTSC field rate to allow use with most 
popular cameras and monitors, (2) LCD technology is mature -- stereoglasses and LCD 
shutters are commercially available from at least two manufacturers, (3) the electronic cir- 
cuitry to synchronize the LCD shutter, camera, monitor, and stereoglasses is inexpensive, 
readily available, and compact, and (4) no special hardware or software is necessary to 
time-multiplex the video signals. 

2. Current Testing and Research 

The SVD has been constructed in the lab and tested on an optical bench with 
mixed results. The SVD does produce a stereopsis effect for most users, but this effect is 
degraded by poor image quality due to camera detector problems, flicker on the video 


monitor, and LCD shutter transmission losses in our system. 

















Additional design challenges include interfacing the SVD with an endoscope for 
future testing, optimization and miniaturization of the synchronization circuit, redesign of 


the LCD shutter, and improvement of image quality. 
B. THESIS OUTLINE 


This thesis contains five chapters and one appendix. Chapter II describes the 
physiological basis of stereoscopic vision including the need for binocular vision, applica- 
tions of a single-camera SVD, and a history of stereoscopic devices. The optical theory 
and SVD design are presented in Chapter III. Results and future development considera- 
tions are detailed in Chapter [V. Finally, Chapter V presents a summary of the research 


performed and some suggestions for follow-on work in the field. 











Il. BACKGROUND 


This chapter presents the fundamentals of visual perception and the physics behind 
human depth perception. A discussion of early stereoscopic devices and trends in 3-D 
video imaging is provided as background for the development of a real-time, single- 
camera Stereoscopic Video Device (SVD). Finally, the applications of a single-camera 


SVD are discussed. 
A. HUMAN VISUAL PERCEPTION IN THREE-DIMENSIONAL SPACE 


The ability to perceive depth through binocular stereopsis over a large field-of- 
view is a visual characteristic that humans share with a only small percentage of the animal 
kingdom. Binocular vision requires that both eyes view the same object simultaneously 
and that their visual fields (each about 170°) overlap to a considerable extent. Although 
this seems rather trivial to humans, most animals have eyes on nearly opposite sides of 
their heads for monocular, or single eyed, panoramic vision and therefore have a very lim- 
ited binocular field of view. Other animals with good binocular vision are cats, predatory 
birds, and primates. [Ref. 8] 

One might wonder how individuals with only one eye can possibly perceive depth 
at all. It turns out that binocular vision is only one of several ways in which humans can 
judge depth; monocular cues such as relative size, accommodation (the muscular effort 
required to focus the eye), shading, and motion parallax (in which the relative motions of 
near and far objects differ) all can be quite helpful in establishing depth. Nevertheless, 
most of these are learned cues which depend upon some reference which is not always 
available. Binocular stereopsis is the only depth cue that is absolute, requires no previous 
knowledge of the object, and requires no relative motion between the observer and the ob- 
ject. Given these facts, it might seem that binocular stereopsis should be by far the most 
dominant of our depth perception cues; but, this is not always the case. Not everyone 
with binocular vision experiences stereopsis; about 2% of those with binocular vision 
have never experienced stereopsis in their day to day lives; another 10% perceive only 


limited stereopsis [Ref. 9]. Also, perception of depth is strongly influenced by cultural and 











physiological background. Studies have shown widely varying stereopsis capabilities 
among individuals, and evidence suggests that practice and experience may play a large 
part [Ref. 8, 9]. One rather amusing example of these influences was reported by Colin 


M. Turnbull in 1961: 


Turnbull was studying the forest pygmies of the Congo basin. His 
guide was a 22-year old pygmy called Kenge, who had spent his whole life 
in the forest, where visibility was limited to a few yards by vegetation. One 
day he left the forest with Turnbull. As they left the forest edge and drove 
across the savanna, a thunderstorm kept visibility to about 100 yards. 
When the rain stopped, some buffalo could be seen grazing in the distance. 
Kenge asked what kind of insects they were. When Turnbull explained that 
they were buffalo, twice as big as the forest buffalo known to Kenge, the 
pygmy laughed and told Turnbull not to tell stupid stories. On being driven 
closer to the animals, he was amazed to see that they were real buffalo, but 
thought Turnbull had used witchcraft to enlarge them. In an environment 
that restricted distance vision, he had never learned to judge size constancy 
at distances. [Ref. 9] 


From this example, it is evident that depth perception is quite complicated, both 
physiologically and psychologically. It depends upon perceptions, visual interpretation, 
memory, and experience. Thus any design for a stereoscopic video device must be com- 
patible with and closely simulate the natural stereopsis process to give good results. For 
this reason, a brief discussion of the physiology of binocular stereopsis follows. 

1. The Physiology of Binocular Stereopsis 

Binocular stereopsis occurs when two views of an object from slightly different an- 
gles are collected by the eyes and fused in the brain. The resulting image developed in the 
mind is three-dimensional. Figure 2-1 shows a horizontal cross-section of the right eye, 
and a plan view of the central visual pathways in the brain. Each eye receives a different 
view of the object and focuses it on the retina, a concave surface at the back of the eye 
lined with nerve cells. Receptors in each retina collect the light and pass the image via op- 
tic nerve fibers to the opposite-side cerebral cortex, the section of the brain responsible 
for vision. As the optic fibers from each eye cross in the optic chiasma, there is an ex- 


change of fibers, and about half of the fibers are routed to one side of the cerebral cortex 





and the other half to the other side. This is the basis behind binocular stereopsis -- each 
side of the cerebral cortex actually receives signals from both eyes; by comparing the lo- 
cation of the respective receptors for each signal (on the retina), a three-dimensional im- 


age is perceived. [Ref 8, 10, 11] 
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Figure 2-1. A Diagram of the Right Eye in Cross-section, and a Plan View of the Central 
Visual Pathway from the Eyes to the Brain [Ref. 11]. 


The problem of fusing the disparate images from the two eyes to form binocular 
vision has been debated since the ancient Greeks. Rene Descartes proposed in the 17" 


century that the optic fibers converged in the brain and were all analyzed together. While 














this was incorrect, it did provide a basis for the unification of the two images in the brain. 
Isaac Newton finally solved the problem in 1704 with his proposal (now known to be cor- 
rect) that there must be an interchange of fibers in the optic chiasma, where the optic 
nerves cross to pass to opposite sides of the cerebral cortex. This proposal laid the foun- 
dation for the concept of binocular stereopsis. The cerebral cortex simultaneously re- 
celves signals from both retinas. These separate images are compared and fused into a 
single image in the cortex. [Ref. 8] 

The following example describes the process of stereopsis. Consider a depth per- 
ception test using two pencils held vertically at different depths from the observer, as 
shown in Figure 2-2. As each eye's fovea (the central high-resolution area of the retina) is 
directed toward the closer pencil, the image of that pencil is projected by the lens of the 
eye onto corresponding receptors in each retina. The two images are said to have zero 
retinal disparity and these signals are fused in the two sides of the cerebral cortex as one 
image with all points at the same depth. When the image of the more distant pencil is fo- 
cused on the retina, the image falls on disparate receptors in each retina. The image from 
one retina is displaced horizontally from that on the other retina. The difference between 
the distances of the retinal images from the corresponding fovea is called the retinal dis- 
parity (this is the distance (d, -d,) shown in Figure 2-2). The cerebral cortex receives the 
signals from the disparate receptors and, if the retinal disparity is not too large, interprets 
the disparity as a depth. [Ref. 8] 

This process depends strongly on the distance of the object from the observer, the 
illumination of the object, and even the interpupillary distance (the distance between the 
pupils) of the observer. Interpupillary distance for most humans is between 50 to 74 mm., 
with the average being about 65 mm. [Ref. 10]. Based upon this fact, and the knowledge 
that the typical human eye can discriminate two objects at no smaller than 30 seconds of 
arc [Ref. 10], it is possible to calculate the limiting range for stereoscopic vision. This 
number turns out to be approximately 500 yards, or about a quarter of a mile [Ref. 11]. 
Although this number is influenced by several factors, it does indicate that stereopsis is 


possible over quite large distances. 
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Figure 2-2. Simple Depth Perception Test. 
This schematic demonstrates how the eye translates a difference in depth of 
two objects into a retinal disparity at the back of the retina. [After Ref. 8] 


2. Flicker and Intermittent Stimulation 

Another parameter which can have a significant influence on the ability to perceive 
binocular stereopsis is the Critical Flicker Frequency (CFF). The CFF is the threshold 
frequency for an intermittent light stimulus above which the eye can no longer perceive 
flicker [Ref. 11]. CFF is of particular importance in the development of alternating-field 
stereoscopic video devices because the presence of flicker in the video image can degrade 
the stereoptic effect. 

One study on the variation of CFF with retinal illuminance shows that CFF actually 
increases with increasing illuminance [Ref. 11]. Figure 2-3 1s a plot of CFF versus retinal 
‘luminance for several different stimulus sizes. From this plot, it can be seen that opera- 
tion of a video device below certain frequencies may subject the viewer to flicker at high 


enough levels of illumination and stimulus SIZe. 
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Figure 2-3. Graph of CFF Versus Illuminance. 
Influence of illuminance on CFF for several test field sizes (measured in an- 
gular degrees swept out by light source as measured from the eye). [Ref. 11] 


Another study found a very slight elevation of CFF with in-phase stimulation of 
corresponding points in the two retinas. Alternatively, out-of-phase (180°) stimuli caused 
a decrease in CFF below that found for one eye alone [Ref. 11]. This is important in the 
design of stereoscopic devices since any alternating-field technique is essentially present- 
ing 180° out-of-phase stimuli to the eyes. A lower CFF is desirable because it lowers the 
minimum rate at which a stereoscopic device must operate to avoid flicker. 

Several studies have been performed to analyze CFF under various conditions: 
however, only two studies could be found on the impact of flicker on stereopsis. These 
studies indicate that binocular stereopsis may not occur during either in-phase or out-of- 
phase stimulation below the CFF. The studies recorded the electrical signals from single 
neural units in the visual system, and found that signals were not present at stimulation 
rates below the CFF [Ref. 11]. For binocular stereopsis to occur, 'binocular' neurons in 
the cortex must be stimulated. Further evidence that flicker does at least degrade stereop- 


sis has been qualitatively shown during experimentation for this thesis. 








3. Creating Stereopsis From a Two-dimensional TV Image 

The ability to perceive a three-dimensional image in the mind through binocular 
stereopsis is truly a spectacular feat. It requires unconscious coordination and feedback 
between the muscles of the eyes, the retina, the cerebral cortex, and much of the rest of 
the brain. In order to better understand the hurdles that must be overcome in developing a 
stereoscopic system from a standard TV monitor, an overview of the stereopsis process 1s 
now presented. This overview ties together the several elements that make up our depth 
perception process to illuminate the importance of such factors as: physiology, flicker, ex- 
perience, learning, and the presence of monocular depth cues. 

For binocular stereopsis to occur, both eyes must first be directed at the same ob- 
ject. This is such a fundamental visual requirement that it occurs without conscious 
thought. In fact, it is not possible for humans to direct each eye independently [Ref. 12]. 
The next step in the stereopsis process is the collection and transmission of the image in- 
formation to the brain. As discussed previously, this takes place mainly in the retina; re- 
ceptors (rods and cones) in the retina relay individual "pixels" of information to the 
cerebral cortex. Other information is also sent: the muscles that control accommodation 
(pointing) of the eye and focusing of the lens also send feedback to the cerebral cortex. 
The cerebral cortex compares the two images from each eye and assesses the retinal dis- 
parity based on the displacement of the receptors receiving the image information. This 
does not end the process. The cortex must also take into account the degree of accommo- 
dation ("toeing-in" or "toeing-out") of the eyes, and the range of the object (partially de- 
termined by the feedback from the lens focusing muscles) to determine the specific depth 
to associate with a given retinal disparity between the images. Once the image is mapped 
in depth, the resulting three-dimensional image is checked for validity. The image is com- 
pared to similar objects in memory for consistency of size and shape; other known objects 
in the field-of-view are used to further establish image depth; and shading, motion paral- 
lax, and other monocular depth cues are employed to verify the overall stereo-image. Any 
one of these monocular depth cues can override the stereopsis effect and compel the mind 


to a different interpretation of the image. 
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This explanation may seem overwhelmingly detailed; yet, it conveys well the com- 
plexity of the process of binocular stereopsis. It also suggests some of the problems that 
may be encountered while attempting to reconstruct stereopsis from the video fields dis- 
played on a TV monitor. These include: (1) flicker -- the eyes do not view the object si- 
multaneously and so stereopsis is degraded or does not occur, (2) accommodation -- this 
valuable depth cue may be distorted or even completely absent when viewing a video 
monitor, and (3) focus -- the eyes are continuously focused at the range of the monitor, 


so this cue is non-existent. 
B. THE IMPORTANCE OF BINOCULAR STEREOSCOPY 


The use of real-time video technology to provide visual feedback for remote op- 
erations is not a new concept. Teleoperations have pervaded all of the science disciplines, 
providing remote visual access to environments too harsh, too small, or too far away for 
actual human presence. Doctors can now use an endoscope with an attached CCD camera 
to perform major surgery with an incision no larger than an inch -- watching the entire 
procedure on a video monitor. Scientists have piloted a robot to the bottom of a volcano 
from a remote location using teleoperations. 

With this growth of teleoperations has come a better understanding of the limita- 
tions of remote operations using only 2-D visual feedback. Many tasks are ideally suited 
for 2-D video since sufficient depth perception cues other than stereoscopic vision enable 
the brain to sufficiently determine relative depth, size, and object configuration so as to al- 
low completion of the task in 2-D. Nevertheless, there are several tasks which are more 
difficult or even impossible without stereoscopic vision. 

1. The Need for Binocular Vision 

It has been argued that binocular stereopsis is of some entertainment value, but is 
unnecessary for most remote teleoperations. After all, people with only one good eye can 
perform many tasks normally considered to require depth perception, such as driving a 
car, flying an airplane, and walking along unfamiliar terrain. In addition, many early stud- 


ies found little experimental evidence of a performance advantage with 3-D displays. 


12 





More recent studies have shown that while binocular stereopsis (or 3-D vision) is not re- 
quired for all tasks requiring visual feedback, it is absolutely essential for some. [Ref. 2,3] 

As a simple example, our strong dependence on binocular depth perception cues is 
evident in tasks requiring ballistic pre-planned motions -- jumping from rock to rock in a 
boulder field, or running downhill on irregular terrain. These tasks are trivial with both 
eyes open, but are nearly impossible with only one eye. The simple task of planning the 
jump and landing spot cannot be performed in either case with only one eye because of the 
absence of monocular depth cues such as perspective or relative size. The requirement for 
absolute knowledge of the distance to the next rock, or of the steepness of the grade, ne- 
cessitates binocular stereopsis. [Ref. 3] 

Another even more fundamental need for binocular vision in humans is discussed in 
Reference 3. In the discussion presented in that paper, the author convincingly argues that 
man's binocular vision evolved because of the critical need for depth perception in a wil- 


derness environment: 


Another task (not commonly encountered by civilized people with 
one eye) is running through unfamiliar forest, where cues of familiar size 
and perspective are missing. All leaves may be the same shape, but their 
sizes are unpredictable, and thus the forest floor and the branches of under- 
growth cannot be gauged in distance. Walking through the forest is not 
too difficult with only 2-D vision, but running requires look-ahead planning 
to see the best pathway through the trees, shrubs and rocks. It is often said 
that binocular cues are not needed because motion parallax provides the in- 
formation. This is only partially true, since motion parallax is very limited 
at the origin of visual expansion (in the direction of travel, where it is most 
needed), and motion parallax .. does not provide all the information 
needed for pre-planning ballistic motor sequences such as running, jump- 
ing, turning left then nght, and so on. [Ref. 3] 


2. Tasks Which Require Stereoscopic Video 

The examples so far have argued for the need for our binocular vision (or that of 
an autonomous or remotely controlled robot) but what about simple teleoperations using a 
remote manipulator -- those missions for which the real-time, single-camera 3-D video 
system might be employed. A task that closely emulates the actual purpose for which 


many stereoscopic systems are designed 1s that of retrieving an object from the center of a 
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mass of tangled wires using a remote manipulator arm and video feedback. A mission to 
maneuver a remote manipulator through a tangle of wire and retrieve an object could be 
encountered in space repair work or undersea salvage. [Ref. 3] 

It was found that this operation was not possible without 3-D vision [Ref 2). 
Subjects participating in the test were unable to navigate the remote manipulator through 
the maze of wire without becoming tangled when using the 2-D video mode. When the 
same task was attempted with binocular vision (using a two-camera system) the task be- 
came quite easy [Ref. 2]. Several other studies on the relative importance of stereoscopic 
video feedback have used the "tangled wire maze" set-up with similar results [Refs. 3, 13]. 

3. Tasks Which are Easier with Stereoscopic Video 

Other operations which commonly use video feedback for remote work have been 
found to be faster and prone to fewer errors when performed with some form of binocular 
stereopsis [Ref. 14]. One common example is the Peg-in-Hole (PiH) task commonly used 
for 3-D versus 2-D comparison studies. In this test, the subjects are required to remotely 
manipulate a grasping device (or robotic arm) to insert pegs in different shaped or sized 
holes using both monoscopic then stereoscopic video feedback. One such test of nine 
trained operators, each with proven depth perception and good visual acuity, indicated 
that, while 2-D visual feedback was adequate to complete the task, the stereoscopic video 
improved both the time to complete the task and the error rate. [Ref. 14] 

Studies of several tasks in which 2-D teleoperations are currently used show vary- 
ing degrees of improvement when stereoscopic techniques are employed. In some areas, 
sufficient depth perception cues other than binocular stereopsis cues may provide suffi- 
cient depth indication, obviating the need for 3-D vision. Examples of this type of applica- 
tion include: simple information gathering, remote manipulations in familiar areas, and 
operations for which known depth cues, such as relative size, shadowing, or perspective 
are present. An overview of several tests comparing 3-D versus 2-D in many teleopera- 
tions fields seems to indicate that 3-D information is most useful in operations performed 
in unfamiliar work areas, where other depth cues are insufficient to provide the necessary 


feedback. [Refs. 3, 14, 15] 
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C. APPLICATIONS OF A REAL-TIME, SINGLE-CAMERA, STEREOSCOPIC 
VIDEO DEVICE 


This section presents several applications of a real-time, single-camera Stereo- 
scopic Video Device (SVD) -- primarily to establish the efficacy of SVD's in fields which 
require near-field, precise remote teleoperations. 

1. Space Applications 

Space missions, by the very nature of the working environment, present a demand- 
ing and rather unique set of challenges to the performance of precise remote operations. 
Spacesuits and life-support connections required for repair operations impose limits on the 
degree of accuracy with which an astronaut can be expected to perform a task on sight. 
This problem is compounded by the complexities that arise when performing even the sim- 
plest tasks in zero gravity. One solution is the integration of remote teleoperations for 
precise remote tasks. NASA has used remote monoscopic, real-time video feedback ef- 
fectively for several years, with perhaps the best know example being the space shuttle 
cargo bay remote control arm. With the establishment of Space Station Freedom, NASA 
will no doubt be forced to increase this reliance on remote teleoperations, possibly even 
controlling tasks from Mission Control. 

The increased demands on video for control and information feedback will inevita- 
bly strain the bounds of conventional Monoscopic Video (MV) technology. As previously 
discussed, some tasks simply cannot be effectively performed using monoscopic video, 
while others are much more difficult and time-consuming than with Stereoscopic Video 
(SV). Why then has SV not been thoroughly integrated into space teleoperations? The 
answer is twofold: (1) the increased complexity and additional space and weight that ac- 
companies current SV systems 1s quite often not sufficiently balanced by the expected 1m- 
provement in time and precision; after all, not all tasks require 3-D vision, and (2) current 
SV systems require two camera detectors (such as CCD's), effectively doubling the band- 
width of the signal that must be transmitted and received. This can be a significant obsta- 


cle to SV use in space, since increasing the signal bandwidth, especially between the 
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spacecraft and mission control, requires more power and may also necessitate upgrading 
existing Communications equipment. 

The solution to the problem of integrating SV into current MV teleoperations sys- 
tems ts compatibility and compactness. The single-camera SVD provides both. As dis- 
cussed earlier, it is compatible with standard NTSC video channels now used in space and 
can be transmitted on a single channel. In addition, it requires no new cameras or moni- 


tors. Also, since it is an in-line optical package, it can be miniaturized quite well for close- 


_ up work in areas in which normal access is limited. Finally, it can be used in both the SV 


and MV mode without any change in the camera or lens configuration. 

2. Terrestrial Applications 

A single-camera SVD is ideally suited to near-field, precision operations and in- 
spections in areas that are normally inaccessible for hands-on work. Although several 
tasks currently fit this description, not all would benefit from SV: an SV system must also 
be cost effective to install and operate, and provide a measurable and significant perform- 
ance improvement over existing MV devices. Examples of applications that meet these 
criteria include: surgical optical devices such as endoscopes [Refs. 16, 17], underwater 
repairs, research, and exploration that employ remote vehicles and robotics [Ref. 3], and 
remote engineering, environmental clean-up, and scientific teleoperations [Refs. 18, 19, 
20]. The common denominator in each of these operations is the absence of relative 
depth cues which would normally de-emphasize the dependence on SV. 

The endoscope is an excellent example of MV teleoperations in an area with few 
strong depth cues -- the human body. The doctor using an MV endoscope must operate 
by watching a 2-D video of the area in which he is working. Because of the high magnifi- 
cation of the image and the relatively small focal length and depth of field of the instru- 
ment, very few significant depth cues are available. As a result, most doctor's experience a 
steep learning curve in controlling the surgical instruments with MV. Even those with ex- 
perience on the device must rely on watching the effect of the surgical instrument on the 


tissue with which it makes contact to establish the position of the instrument. 
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An in-line optical device such as the one proposed in this thesis can be packaged to 
attach to a standard endoscope (6-12 mm). In fact, one of the design criteria for the de- 
velopment of the prototype SVD is the construction of a device small enough to fit on the 


end of an endoscope. 
D. A SURVEY OF STEREOSCOPIC VIDEO TECHNOLOGY 


1. Stereoscopy in Entertainment 

By the end of the 18" century, advances in the study of physiological optics and 
the proliferation of relatively inexpensive optical equipment had spawned an interest in 
stereoscopic devices. Charles Wheatstone developed the first stereoscope in 1834. This 
device used two mirrors mounted at a 45° angle to each eye to present separate but similar 
views of the same picture to either eye. Although this device worked quite well, it was 
not an overnight success; the "pictures" that Wheatstone used were merely hand-drawn 
geometric shapes -- photography as we know it had not yet been invented. [Refs. 10, 21] 

For the general public to become excited about a stereoscopic video device, the 
image produced had to be somewhat more real and exciting; this problem was soon 
solved by the invention of the photograph. Soon after that, David Brewster invented a 
stereoscope using two plano-convex lenses to view stereoscopically-related photographs. 
His stereoscope is shown in Figure 2-4. This device was displayed at the London Exposi- 
tion in 1851, and within 3 months, over 250,000 were sold. [Ref. 21] 

The final ingredient necessary to bring the science of stereoscopy to the general 
public was the invention of the motion picture. In 1893, Edison and Lumiere simultane- 
ously developed a camera with intermittent movements that used an endless band of cellu- 
loid with a light-sensitive emulsion. A combination of Lumiere's frame rate (16 fps) and 
Edison's film size (35 mm) soon became the industry standard. Lumiere went on to de- 
velop the first stereoscopic motion picture, using a blue dye for the right image and an or- 
ange dye for the left image. The audience viewed the film with blue and orange glasses, so 
that the blue side could only see the orange image, and the orange side only the blue im- 


age. [Ref. 21] 
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Figure 2-4. Charles Brewster's Stereoscope. 
The stereogram is inserted in a slit in the side (shown fully inserted). The 
stereogram, consisting of two stereoscopically-related i images, 1S situated 
inside the focal plane of the eyepiece lenses so that the two images are su- 
perimposed at the common focal plane to produce a 3-D | image. [Ref. 10] 


2. Stereoscopy in War 

One of the first applications of stereoscopy for war was the development of a 
stereoscopic aerial camera used for reconnaissance during WWII. General George W. 
Goddard headed up a special division in the Air Force created to study how stereo-aerial 
photographs could be used to provide strategic information. Goddard's division designed 
a special stereo-camera called "The Sonne Strip Camera" -- a low altitude stereo camera 
that used standard nine-inch aerial film. Although the stereo-camera was used only spar- 
ingly during WWII, it was brought out of mothballs for use in 1962. The Cubans had es- 
tablished nuclear missile bases, and President Kennedy wanted undeniable proof that the 
missiles were ready to be launched. So, he had the Sonne Strip camera mounted on the 
wing of a high speed jet. The jet made a low pass over Cuba at maximum speed, and Ken- 
nedy had his proof. The Sonne Strip camera and some of these photographs can now be 
seen at the Air Force Museum in Dayton, Ohio. [Ref. 21] 

3. 3-D Video Technology 

3-D video technology refers to the use of a camera or system of cameras to make a 


video image which, when viewed with special glasses or a special video monitor, produce 
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a 3-D effect. Two general categories of 3-D video technology are currently in use: (1) 
stereoscopic, those requiring use of special glasses or viewing aids, and (2) autostereo- 
scopic, those which require no glasses -- many of these systems produce a pseudo- 
stereoscopic effect but do not employ binocular stereopsis. 

a. Stereoscopic Video Systems 

Most 3-D television systems use two cameras and a system of hardware or 
software to time-multiplex the fields at either the standard NTSC field rate, or twice that 
rate. The cameras are oriented such that the angular disparity between the views is com- 
parable to that produced by the offset of the eyes. A 3-D image ts realized when the sub- 
ject views the monitor through shuttered glasses, oppositely polarized lenses, or some 
other alternating-field technology. Of these systems, Stereographics® has one of the most 
innovative and refined designs. [Ref. 5] 

Figure 2-5 shows the layout of the Stereographics® 3-D video system. The 
system consists of two standard cameras fixed such that their optical centerlines are 2-1/2 
inches apart. these cameras feed the video signals to a controller at a field rate of 60 fields 
per second (fps) each. The controller stores the two signals and alternately displays the 
fields on the monitor at a rate of 120 fps. The monitor 1s viewed with Stereographics® 
CrystalEyes LCD glasses, which are synchronized to the field rate of the monitor using a 


CrystalEyes infrared (IR) emitter [Ref. 5]. 
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Figure 2-5. Stereographics® 3-D Video System. [After Ref. 5] 


This system presents the most viable product presently on the market for real-time 


stereotelevision. As with any two camera system, this design does have some problems: 
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two channels are required for transmission and reception of the signal, and the system re- 
quires a special controller unit and a high speed (120 fps) video monitor. 

Several other designs of stereoscopic video systems using mechanical shut- 
ters, color filters, polarizers, and lens-prism devices have been developed. Figure 2-6 
shows a single channel, spatial multiplexing system which uses a standard video monitor 
with a specially polarized stereo-screen. The images are spatially separated by the lens 
system in front of the camera. The video monitor displays both images side-by-side or 
over-and-under, and the stereo-screen polarizes the light from the two images in opposite 
directions. The glasses include oppositely polarized lenses for each eye such that each eye 
sees a separate view. Problems with this system include: loss of image resolution, a de- 
crease by 1/2 in image size, and possible visual distortion and head viewer orientation limi- 


tations. [Ref. 6] 
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Figure 2-6. Single-channel, Spatial Multiplexing System. [After Ref. 6] 





Other devices use mechanical shutters or specific color filters (usually red 
and cyan) at both the camera lens and the monitor, as displayed in Figure 2-7. These de- 
vices must also be used with special glasses to perceive stereopsis. 

The systems described above are only a representative sample of the types 
of systems that have been developed to produce binocular stereopsis. While each uses 
somewhat different methods to obtain separate, stereoscopically related left and right im- 


ages, all of these devices use true binocular stereopsis to form the 3-D image. 
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Figure 2-7. Single-camera, Color-multiplexing Scheme with Colored Glasses. 
[After Ref. 6] 


b. Autostereoscopic Video Systems 
Autostereoscopic systems have been developed for both individual use and 


group viewing. Figure 2-8 shows both of these systems. In the individual system, two 
cameras are used to present related images to the eyes through a special screen. The 
viewer must maintain his head at the optimum viewing position to detect the stereoscopic 
effect. The group system uses a lenticular screen assembly to provide several possible 
viewer positions. The primary disadvantage of these systems is quite obvious, the viewer 
must maintain his head within fairly tight bounds. [Ref. 6] 

The VISIDEP™ (visual image depth enhancement by parallax induction) 
alternating-frame technology is a more recent technology which produces limited depth 


enhancement by shuttering left and right views of an object on the monitor at a rather slow 
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Figure 2-8. Autostereoscopic Stereo-pair Systems. [After Ref. 6] 
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rate (usually about 10 Hz). The parallax between the two views permits a judgment of 
relative depth [Ref. 16]. Because of the slow speed at which the alternate views are pre- 
sented in this method, both eyes see each view -- thus no real binocular stereopsis occurs. 
Also, the slow shutter rate can cause dizziness and headaches if viewed for prolonged 


periods. 
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III. DESIGN OF A 3-D VIDEO IMAGING DEVICE 


This chapter documents the development of a real-time, single-camera, stereo- 
scopic video imaging device. First, the theory behind the concept of a single-camera 
stereoscopic video device (SVD) is described in detail. Next, the development of the ini- 
tial prototypes is reviewed, including a discussion of the theory of operation of each com- 
ponent and a description of the modifications required to attain the final SVD design. 
Lastly, the design for a working SVD, comprised of a lens system, an LCD shutter, and a 


synchronizing circuit, is presented. 
A. ALTERNATING-FIELD 3-D VIDEO THEORY 


Several SVD designs make use of the alternating-field concept together with some 
type of shuttering glasses to produce a stereoscopic effect. These SVD's have many fea- 
tures in common: two stereoscopically-related views of an object must be taken, either si- 
multaneously or alternately at a fast rate; the views are alternately displayed on a video 
monitor at the field rate of the monitor; and, finally, the monitor must be viewed with 
some shuttering device such that only the left eye sees the left-hand image and only the 
right eye the right-hand image of the object. How our SVD will achieve these steps is dis- 
cussed in this section. 

1. Lens Theory 

Recall that a simple lens focuses parallel rays at its focal point, as in Figure 3-1. 
Parallel rays incident upon a lens from a certain angle 8, as measured from the centerline 
of the lens, are focused at a height y above the centerline at the focal plane of the lens. A 
detailed discussion of geometric optics and ray tracing can be found in Reference 22. 

An object reflects light such that rays emanate in all directions from every point on 
the object. One method of accounting for these rays is to group them according to the an- 
gle at which they leave the object. Those rays leaving the object at the same angle are 


parallel. Rays which leave the object at the same angle with respect to the centerline of 
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Figure 3-1. Simple Lens Analysis Using Ray Tracing. 


the lens system will be focused to the same point on the focal plane. Rays which leave the 
object at a different angle will be focused at a different point on that same focal plane. 
Parallel rays emanating from the object to the left of the centerline are focused on the left 
side of the focal plane of the lens, and those emanating from the right are focused on the 
right side of the focal plane. This concept can be used to spatially separate different angu- 
lar views of the object, and the separate views can then be blocked to allow only one view 
at a time to the camera detector. 

2. Lens System Description 

Figure 3-2 is a top-view depiction of a general alternating-field stereoscopic video 
device. Parallel rays from the object are focused by the objective lens at the focal plane of 
the lens. To get only one view of the object, we must b/ock all those rays which emanate 
from the object at an angle to one side of the centerline of the lens system (e.g., those rays 
which make up the left-hand view of the object) and we must pass those rays which ema- 
nate from the object to the other side of the centerline (e.g., those rays which make up the 
right-hand view of the object). To do this, a shutter is used to alternately block the rays 
from the left-hand and then the right-hand side of the image. This shutter must be prop- 


erly synchronized with the camera, the monitor, and the stereoglasses. 
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Figure 3-2. Overhead View of a General Single-camera Alternating-field Device. 


The rays from a single view are re-collimated by the second lens in order to re- 
form the image. Then, the image 1s focused by the camera's objective lens on the camera 
detector. The detector collects the image and transmits it to the monitor for display as 
one video field. 

3. Producing a Stereoscopic Video Image 

A standard NTSC TV monitor displays one frame every 33.33 msec (a frame rate 
of 30 Hz). Normally, a video image, or frame, consists of two interlaced video fields. For 
Our purposes, we will use a single non-interlaced field as our image. The two 
stereoscopically-related views are alternated on the monitor at the field rate of 60 Hz. In 
order for the viewer to perceive a separate view of the object with each eye, he must wear 
stereoglasses. These glasses use an LCD shutter in each lens to pass light to only one eye 
at a time at a rate set by a special synchronization circutt. 

4. Summary of Alternating-field Video Stereoscopy 

An objective lens is used to focus rays from the object to the focal plane of the 
lens. Here the rays are filtered by a shutter such that only one view of the object is passed 
at a time. The light rays that are passed by the shutter are collimated to re-form the im- 
age, then focused on the camera detector by a second set of lenses. The image incident 
upon the detector is transmitted to the video monitor and displayed as one video field (for 
1/60 second). Finally, the LCD stereoglasses, operating in synchronization with the shut- 
ter, camera detector, and monitor, allow only one eye to see the video image, the left eye 


seeing only the left view and the right eye only the right view. If this process is performed 
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fast enough, at a frequency above the critical flicker frequency (CFF) defined in Chapter 
II, the brain will be able to fuse the disparate images from each eye into a three- 


dimensional image of the object. 
B. ELECTRO-MECHANICALLY SHUTTERED STEREOSCOPIC SYSTEMS 


Three prototypes were employed during the development and testing of the single- 
camera SVD. The first system consisted of a chopper wheel for both the initial shuttering 
_ of the two views of the image at the focal plane and for viewing the video image on the 
monitor. This device was designed and built at the Naval Postgraduate School by Profes- 
sors Robert Keolian and David Cleary of the Physics Department. In the second system 
we used stereoglasses instead of a chopper wheel to view the video monitor. These two 
systems will be discussed in this section. The final design implemented a zoned LCD shut- 
ter at the focal plane of the lens device. This design will be discussed in the next section. 

1. Chopper Wheel Shuttering System 

A schematic of the first shuttering system is shown in Figure 3-3. The separate 
views of the object are shuttered at the focal plane by a chopper wheel driven by a DC 
motor. The chopper wheel is a transparent plastic disk with opaque areas configured to 
intermittently block the light as the wheel rotates, as depicted in Figure 3-3. A similar 
chopper wheel located between the monitor and the viewer shutters the view to each eye. 

Each chopper wheel DC motor is synchronized to the video signal using a lock-in 
amplifier. A sketch of this circuit is shown in Figure 3-4. The reference signal to the 
lock-in amplifier is a square wave synchronized to the video signal. The feedback signal 
(or comparison signal) to the lock-in amplifier is the output of a photodetector which 
senses motor frequency and phase from a white and black pattern connected to the motor 
shaft. The output of the lock-in amplifier is first amplified and then offset with a DC gain 
to raise the speed of the motor to that required for synchronization. 

The system was constructed on an optical bench and tested. All participants re- 
ported a distinct binocular stereopsis effect when viewing the test object. This was con- 


sidered a successful test; however, several features of this device are unsuitable for the 
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Figure 3-3. Top View Schematic of the Chopper Wheel Shuttering System. 
Each chopper was a transparent plastic wheel with opaque (black) areas 
zoned to intermittently block the light as the wheel rotated. An individual 
DC motor driven by a synchronizing circuit turned each shutter. 


intended applications of the SVD, particularly those requiring miniaturization. These un- 
suitable features include the following: (1) both chopper wheels are too large for the 
SVD's intended applications, and the video monitor chopper wheel cannot be miniaturized 
without a deterioration of the viewing conditions, (2) the video monitor chopper wheel 
spinning near the observer's face is objectionable and severely limited the viewer's mobil- 
ity, and (3) the power required by the DC motors is too high for efficient use in a smaller, 
more portable system. Despite these problems, the test did provide a proof-of-concept for 
a single-camera SVD. 

2. Chopper Wheel Shutter with Stereoglasses 

The next step towards the development of a practical system was to replace the 


chopper wheel for the monitor with LCD stereoglasses. An example of commercially 
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Figure 3-4. Schematic of DC Shutter Motor Control Circuit. 


available stereoglasses for this application is the CRYSTALEYES model developed by 
Stereographics®. The stereoglasses used for this thesis shutter the light to each eye such 
that only one eye is receiving an image from the video screen at any one time. The stereo- 
glasses are synchronized to the video signal by an IR emitter which transmits a timing sig- 
nal to a receiver on the glasses. 

The chopper wheel shutter system with stereoglasses is shown in Figure 3-5. The 
chopper wheel in this lens system is identical to that discussed in the previous section, and 
the control circuit for this chopper motor is similar to that shown in Figure 3-4. A de- 
tailed description of the glasses, including the modifications necessary to work in our sys- 
tem, is discussed in more detail below. 

This system as shown in Figure 3-5 was tested on an optical rail, and binocular 
stereopsis was again perceived by several viewers. The stereoglasses were convenient to 
wear and provided the viewer with more mobility than was possible with the stationary 
chopper wheel shutter. Nevertheless, the use of a chopper wheel shutter in the lens as- 


sembly still did not meet all of the design objectives for the prototype. In particular, the 
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chopper wheel draws too much power and is not easily scaled down to the sizes necessary 
for the intended applications of the SVD. A further refinement to the SVD design was 


needed to address these shortcomings. 
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Figure 3-5. Top View of Chopper Wheel Shutter Device Used with Stereoglasses. 


C. LCD SHUTTERING SYSTEM WITH STEREOGLASSES 


The final single-camera SVD design uses an LCD shutter in place of the chopper 
wheel in the lens assembly. Figure 3-6 is a schematic of the final SVD. The major com- 
ponents of the design are the lens assembly, the LCD shutter assembly, the stereoglasses, 
and the synchronizing circuit. The device is designed to work with a standard camera and 
video monitor operating at the NTSC field rate. The design is compatible with a commer- 
cial endoscope to provide a realistic application with which to test the device. This sec- 


tion describes the design of each component of the final system in detail. 
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Figure 3-6. Schematic of the Single-camera, Stereoscopic Video Device. 


1. Lens Assembly 

The lens system was designed for close-up, narrow depth-of-field operations. The 
objective lens and the collimating lens are achromatic, 25 mm diameter lenses with an ef- 
fective focal length of 50 mm. A schematic of the lens arrangement is shown in Figure 
3-6. Achromatic lenses were chosen both to allow compatibility with color cameras and 
to provide the highest precision focus for both black-and-white and color cameras over the 
entire range of the visible light spectrum. Other lens combinations are available for situa- 
tions requiring a larger depth-of-field or a reduced viewing angle. 

2. LCD Shutter Assembly 

Because of the fast response, controllability, and compactness of liquid-crystal 
shutters, it was determined that LCD technology offered the best alternative to a chopper 
wheel shutter. LCD's have been developed that switch within microseconds, and the con- 
trol circuit for an LCD typically consists of only a single voltage signal and a ground line. 


Furthermore, LCD's can be miniaturized to the sizes required for this project. 








a. LCD Shutter Theory 

A general design for an LCD is shown in Figure 3-7. A plane polarizer 
placed before the liquid-crystal (LC) polarizes the incident light in one direction. The LC 
layer rotates the plane of polarization of the light by 90° when the LCD electric field ts 
OFF (deenergized). When the LCD ts energized, the liquid crystal layer does not rotate 
the plane of polarization of the light. By placing a second plane polarizer after the LC, 
with its polarization axis oriented at a right angle with respect to the first polarizer, a light 
shutter is formed. The second polarizer will only pass the light that has been rotated by 
90° with respect to the initial direction of polarization. In other words, it will only pass 
light when the LCD is deenergized. By applying and removing the electric field on the 


crystal, the polarizer-LC-polarizer system acts as a light switch or shutter. [Ref. 23, 24] 





Fieid On 


Field Off 


Figure 3-7. A Sketch of a Light-twisting LCD Shutter. 
Thread-like liquid-crystal molecules are arranged so as to rotate the polari- 
zation of light. Non-energized pixels enable the light to pass a second po- 
larizer (left). When a voltage is applied, the twist is disrupted, and the light 
is blocked (right). [After Refs. 23, 24] 


b. LCD Shutter Design 


The LCD shutter design for use in the single-camera 3-D device consists of 


a circular LCD shutter with two independently controlled active LCD areas and a non- 


active center section. The LCD shutters were designed with semi-circular active zones to 
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Figure 3-8. Schematic of the LCD Shutter, Showing Both States of the Device. 


allow the maximum light from each view through to the detector. The center section of 
the shutter contains liquid-crystal material but is not controllable, since these rays are es- 
sentially at a zero angle of incidence and are common to both views. 

Figure 3-8 shows the design of the LCD shutter. A photograph of the 
LCD shutter device, including the polarizers, focal plane mask, and casing, is shown in 
Figure 3-9. Light passes through a plane polarizer, then through a focal plane mask (dis- 
cussed in detail in the next section), into the LCD shutter, and through a second polarizer. 
The LCD shutter and the mask are located at the focal plane of the objective lens. Also, 
the collimating lens is situated at exactly one focal length behind the shutter. 

To minimize the flickering of the image on the video screen, it is desirable 
to use a shutter in the lens system which switches within about 100 microseconds. This 
minimizes the fraction of the image duty cycle taken up in switching the LCD shutter. To 
meet this switching time, a ferroelectric LCD shutter produced by Boulder Nonlinear De- 
vices was used. The LCD shutter is reported to have a switching time as fast as one mi- 
crosecond, a transmission ratio of nearly 1000. The transmission ratio is the optical power 
transmitted when LCD is ON (deenergized) relative to power transmitted when LCD is 
OFF (energized). The device also exhibits a good transmission characteristic over most of 
the visible light region. Figure 3-10 is a graph of the transmission characteristic of the 
LCD shutter over the visible light band. The drop in transmission at the upper and lower 


ends of the visible spectrum represent light loss in the system. 
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Figure 3-9. Photograph of the LCD Shutter Device and Casing. 
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Figure 3-10. Computer Simulation of the Transmission Versus Wavelength Characteristic 
for the Zoned LCD Shutter Used in the Stereoscopic Video Device. 
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c. Focal Plane Mask 
An SVD using the LCD shutter described above was fabricated and tested. 


The resulting video image was blurry, causing a degradation of the stereopsis effect. The 
cause of the blurring was determined to be the wide range of views of the object presented 
to each eye. In normal binocular stereopsis, each eye perceives only a small range of 
views of the object, as determined by the solid angle bisected by the pupil of the eye. 
However, in our original device design, the camera received a range of views only limited 
by the size and characteristics of the lens and shutter system. The presentation to each eye 
of this wide range of views of the object made the image appear blurry, thus inhibiting the 
stereopsis process. 

The solution was to design a mask which would be placed at the focal 
plane of the lens. The position of the mask with respect to the shutter mechanism and the 
shape of the mask can be seen in Figures 3-6 and 3-9. This design was chosen to approxi- 
mate the solid angles intercepted by the eyes for the given lens system. The system was 
retested using the mask and it was found to provide sufficient filtering to eliminate the 
blurring of the image. 

3. Integrated Lens and Shutter Assembly 

Figure 3-11 is a photograph of the prototype device on an optical rail. The system 
consists of an objective lens, a focal plane mask, the LCD shuttering mechanism (including 
polarizers) and the collimating and focusing lenses. For integration with an endoscope, 
the entire device as shown on the optical rail must be fitted into a single, lightweight cas- 
ing that can be attached to the end of the endoscope. The device requires only three elec- 
trical leads: one control line for each zone of the LCD shutter and a common ground line. 
The operation of the SVD is not sensitive to a rotation of the device about the optical axis, 
since any orientation will provide two views of the object. 

4. Stereoglasses 

The lenses in the stereoglasses are made up of a liquid-crystal material sandwiched 
between two plane polarizers. The LCD lenses (or shutters) are driven by a battery pow- 


ered circuit with a proprietary, oscillator-driven integrated circuit (IC) chip controlling the 
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Figure 3-11. Photograph of Electro-optical Shuttering Device on an Optical Rail. 
On the far right is the object being viewed, then from right to left are the objective lens, 
the encased LCD shuttering device, the collimating lens, a focusing lens, and the camera 


timing. The glasses are synchronized with the monitor via an IR emitter, usually located 
near the monitor, and an IR photodetector on the front of the glasses. The emitter IS 
driven by a square wave derived from the camera signal, as described below. 

The stereoglasses used in this thesis were designed for use with a two-camera sys- 
tem as discussed in Chapter II. Recall that for a two-camera system, a special monitor op- 
erating at 120 fps was viewed with the glasses, shuttering at 60 Hz. Our system required 
stereoglasses that operated at 30 Hz. It was determined that our off-the-shelf stereo- 
glasses would not operate at this frequency. The glasses would not stay locked-in at the 
proper phase and frequency below about 32 Hz. In order to correct this problem, the in- 
ternal circuitry of the glasses had to be modified. The glasses have a crystal oscillator that 
is used in the timing circuit. The factory-installed oscillator had a frequency of 32 kHz. 


We replaced this with a 30 kHz oscillator and found that the glasses could then operate at 


30 Hz. 














5. Synchronizing Circuit 
The single-camera, stereoscopic video device consists of a lens and shutter assem- 
bly, a video camera and monitor operating at the standard NTSC rate, and a pair of 
stereoglasses for the viewer. These components must operate synchronously with the 
camera in order to allow the viewer to perceive three-dimensions; thus, a synchronization 
circuit was developed to control the frequency and phase at which both the LCD shutter 
and the stereoglasses operate. | 

| A block diagram of the synchronizing circuit and the control signals and timing sig- 
nals developed in the circuit are shown in Figure 3-12. The video signal from the camera, 
shown as timing diagram (a) in Figure 3-12, is sent both to a video synch-separator chip 
and to the video monitor. Each shaded block on this graph 1s a single video field, or, in 
our case, a single view of the object. The video synch-separator chip extracts a timing sig- 
nal from the video signal and then outputs a 30 Hz TTL square wave at the frame rate of 
the video signal, as shown in timing diagram (b). This signal is called the "odd/even field 
output” of the synch-separator chip. The odd/even field output is then sent to the IR emit- 
ter for the stereoglasses to synchronize the glasses with the camera and monitor. This sig- 
nal is also sent to a delay and duty cycle shift circuit which produces a signal that is both 
delayed in phase and modified in duty cycle. The phase delay and duty cycle can be var- 
ied, and one example of the outputs from this circuit is shown in timing diagrams (c) and 
(d). The shifted TTL signals are then amplified and sent to the two independently con- 
trolled zones of the LCD shutter as control signals. The control signal for the mght zone 

of the LCD shutter is shown in timing diagram (e). 

a. Synchronization of the Stereoglasses with the Monitor 

The stereoglasses were tested using the odd/even field output of the synch- 
separator chip to drive the IR emitter. It was found that with this signal, the stereoglasses 
were in-phase with the video monitor, and so no delay or duty cycle modification was re- 
quired for the glasses. When viewing the video monitor with glasses that are not in-phase 


with the monitor, a black horizontal line will appear on the screen. This line indicates the 
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Figure 3-12. Block Diagram of Synchronization Circuit (top), and Timing Diagrams 
Showing the Control Signals Developed by the Circuit (bottom). 
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point at which the old image is fading and a new image is being illuminated on the moni- 
tor. When viewing the monitor with stereoglasses that are in-phase with the monitor. the 
black line 1s not visible. 

b. Synchronization of the LCD Shutter with the Camera 

Once the timing signal is produced, a delay and/or a duty cycle modifica- 
tion may have to be introduced to ensure that the LCD shuttering device is properly syn- 
chronized with the camera detector. The LCD shutter is phased to the camera detector to 
optimize the individual image clarity by passing only one view to the detector over one im- 
age collection field. 

We used a programmable microcontroller to delay and modify the duty cy- 
cle of the signal from the separator chip. The microprocessor produces a delay that can be 
varied from 1.4 to 16.3 msec using an 8-pin dipswitch (256 possible delays, 58.2 usec 
each). The output of the delay circuit is fed into a duty cycle modification circuit. A 
variation in the pulse width of the square wave from 5 to 50% can be accomplished with 
this circuit. 

The controller is a Parallax PIC 16C54, 18-pin microcontroller. The mi- 
crocontroller is programmed in a proprietary object code on an IBM compatible personal 
computer. This object code is a near-assembly language instruction set which is then as- 
sembled to machine code and loaded into the microcontroller. [Ref. 26] 

The programs for both the variable delay circuit and the duty cycle modifi- 
cation circuit are included in the Appendix. They were developed by Eric Conner of the 
NPS Physics Department. The circuit takes the TTL square wave output of the synch- 
separator chip and produces two outputs: (1) a square wave with a modified phase and 
pulse width, and (2) a second square wave identical to (1), but shifted in phase by 180°. 
Two outputs are necessary to provide a signal to each active zone of the in-line LCD 
shutter. 

The final synchronizing circuit allows synchronization of the stereoglasses 
and both zones of the LCD shutter to the video monitor and camera detector. With the 


variable time delay and duty cycle, the SVD can be tested with various lighting conditions, 
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cameras, monitors, and lens configurations. The results of these tests are the subject of 


Chapter IV. 
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IV. RESULTS 
A. TESTING OF THE PROTOTYPE STEREOSCOPIC DEVICE 


The testing of each of the prototype SVD's developed for this thesis produced 
mixed results. A stereoscopic effect was evident with both the mechanically shuttered 
prototype and the electro-optically shuttered prototype, but the stereoscopic effect was 
only weakly discernible in most practical tests. The three-dimensional effect was verified 
by both student volunteers and NPS faculty members in a qualitative test of the device. 

A simple depth perception test was also used to check for an improvement in 
depth perception with the SVD. The test consisted of viewing identical objects that were 
slightly offset. The objects were viewed on the video monitor both with and without the 
SVD. All of the participants failed to distinguish the proper relative depth between the 
objects when no SVD was used. With the SVD, three out of four subjects were able to 
positively establish the relative depth of one object with respect to the other on several tri- 


als. One subject reported seeing no three-dimensional effect. 
B. PROBLEMS ENCOUNTERED DURING TESTING 


The achievement of stereoscopic vision from a single-camera video device, while 
significant, did not accomplish all of the objectives of the research. One of our goals was 
the development of an SVD that could meet realistic teleoperations demands in the space 
environment. The prototype SVD has not yet met these requirements. The following 
problems arose in preliminary testing which must be solved prior to any in-depth, func- 
tional testing of the device can be performed: (1) shadowing of the image, and (2) flicker 
of the video image. 

1. Shadow Images in the Video Picture 

The main problem with the current SVD is the occurrence of shadowing. Shad- 
owing is the appearance of a faint image of the alternate video field when viewing the 
other field on the monitor. Shadowing could be due to a long time constant for the phos- 


phor decay on the monitor, or to improper synchronization between the LCD shutter and 
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the camera. Improper synchronization of the stereoglasses was ruled out based on the 
synchronization tests for the glasses discussed in Chapter III. Based upon these same 
tests, we do not believe the problem is the monitor either. A sufficient phosphor decay 
rate 1s evident in the fact that the black horizontal strip is present on the monitor when the 
glasses are out-of-phase with the stereoglasses. This strip shows the fading of the previ- 
ous field prior to the next field being scanned onto the screen. 

In order to determine the cause of the shadowing, tests were performed using a 
strobe light as the sole illumination source for the object being viewed. The strobe light 
was driven by the synchronizing circuit at 30 Hz such that the strobe light was in-phase 
with the rising edge of the square wave synchronizing pulse as verified by an oscilloscope. 
The testing of the SVD using the strobe light revealed that the video camera was transmit- 
ting an illuminated view for both the in-phase field and the out-of-phase field. This shad- 
owing could not be removed at any setting of the variable time delay and duty cycle 
modification circuit controlling the LCD shutter. This indicates that the camera has an in- 
tegration time which is incompatible with the LCD shutter. A second camera was tried 
with similar results. More work is required to understand the shadowing problem. 

2. Flicker in the Video Picture 

The second problem that arose when testing the prototype SVD was the appear- 
ance of flicker. As discussed in Chapter II, flicker in the image at a rate below the critical 
flicker frequency (CFF) may cause a degradation or complete loss of the stereopsis effect. 
The amount of this degradation is difficult to quantitatively determine because of the pres- 
ence of shadowing in the image. 

The CFF is dependent on several test-specific variables, such as the illumination in- 
tensity on the object, the object contrast, the solid angle bisected by the object at the eye, 
and the specific image and timing characteristics of both the camera and video monitor. 
Additionally, CFF varies from person to person over a relatively large range of frequen- 
cies. [Ref. 11] For these reasons, specific values of the CFF for the various test condi- 


tions were not determined. 
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Stereopsis was perceived by several test subjects during preliminary testing of the 
SVD. This suggests that the SVD is indeed operating at a frequency slightly above the 
CFF. Nevertheless, the presence of flicker was reported as objectionable by some sub- 
jects. Again, this problem is coupled to the problem of shadowing in the image. Further 
testing of the influence of flicker on the stereopsis process must be performed when the 
shadowing problem ts resolved. 

3. Transmission Losses Through the Shuttering Mechanism 

One further problem which arises in the use of LCD shutters for optical video sys- 
tems is the 50% to 60% transmission loss in the shuttering system in the transparent state. 
This is primarily due to the fact that the light must be plane polarized prior to entering the 
LCD shutter (this accounts for slightly over 50% of the transmission loss). The additional 
10% loss in transmission is due to the chromatic selectivity of the particular LCD shutter. 

The transmission characteristic of the LCD 1s shown in Figure 3-10 in Chapter IIT. 
While the shutter itself passed nearly 100% of the incident polarized light at about 530 nm, 
this transmission dropped to 50% at the near-ultraviolet end of the spectrum (420 nm) and 
to 80% at the near-infrared end of the spectrum (690 nm). The average loss of light due 
to the LCD shutter over the visible band is approximately 10%. 

Although the 50% loss of light due to the plane polarizer is not recoverable in this 
SVD, the 10% loss due to chromatic selectivity in the LCD shutter can be significantly re- 
duced. An achromatic LCD shutter, which has transmission near 100% over the entire 


visible region of the light spectrum, is available through Boulder Nonlinear Devices. 
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V. CONCLUSIONS & RECOMMENDATIONS 


A. CONCLUSIONS 


A real-time, single-camera, stereoscopic video device was developed for use with a 
standard video camera and monitor. The device uses a lens system and LCD shuttering 
mechanism to present alternating views of the object to the camera detector. The alternat- 
ing views are presented to the video monitor at the standard video field rate. Specially 
modified stereoglasses allow each eye to perceive every other video field as displayed on 
the monitor. The result is that only one view of the object is presented to each eye, which 
permits the perception of depth in the brain through the process of binocular stereopsis. 

Testing of the prototype stereoscopic device on an optical rail produced a mild 
stereoscopic effect in most cases. One test subject reported no stereoscopic effect. The 
quality of the image was poor due to transmission losses through the LCD shuttering 
mechanism, flicker apparent in the video image, and shadowing of the alternate view onto 
the image on the video monitor. Testing indicated that the source of the shadowing prob- 
lem was the camera detector. Future testing of the SVD with a higher quality camera de- 


tector 1s required. 
B. RECOMMENDATIONS FOR FUTURE RESEARCH 


The presence of a stereoscopic effect using the prototype SVD has been shown. 
Furthermore, the quality of the image has been improved by the addition of the focal plane 
mask and the time delay to the LCD shutter. The next step in the development of a func- 
tional stereoscopic video device is to test the SVD on a specific device, such as an endo- 
scope. The synchronization of the LCD shutter is dependent upon the characteristics of 
the particular camera detector used. Thus, once an endoscope detector is available, fur- 
ther experimentation can be performed to determine the optimal phase delay and duty cy- 
cle for the LCD shutter. With these results in hand, the synchronizing circuit can be 
simplified (since the time delay and duty cycle can now be fixed with one microcontroller 


chip) and miniaturized for integration with the shutter device. 


45 














Further research in ways to remove the shadow image from the video picture 1s 
also necessary. The problem may be resolved with the use of a higher quality camera de- 
tector. With the shadow image removed, further testing can be performed to determine 
whether the flicker in the image is causing significant degradation in the stereopsis effect. 
One possible solution to the flicker problem is the move to a high speed camera and moni- 


tor. This option would require specialized equipment and may limit the advantages inher- 


ent in the SVD as designed. 


Additional testing of the SVD on a larger control group is not recommended until 
the shadowing problem is resolved. The testing of the device on a practical level must be 
performed at the optimum time delay for the LCD shutter mechanism to obtain a substan- 
tive evaluation of the future potential for the SVD. If the shadowing on the video monitor 
can be removed, a more distinct stereoscopic effect may be present. This will allow the 
determination of the optimum phase delay of the LCD shutter. 

The testing performed for this thesis used a lens system which was developed for 
close-up viewing applications. Because of the objective lens size (25 mm in diameter) and 
the small depth-of-field of the system, the size of the object that could be viewed was lim- 
ited to less than 25 mm. In order to view a larger object from the same viewing angles, 
the diameter of the objective lens in the SVD must be increased. To view an object at a 
greater distance from the device, a lens system with a larger focal length is required, and 


the focal plane mask may need to be adjusted to capture the desired viewing angles. 
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APPENDIX. COMPUTER CODE FOR SYNCHRONIZING CIRCUIT 


The computer code for the time delay and duty cycle modification circuit 1s written 
in a proprietary code developed by Parallax, Incorporated. The code ts compiled into as- 
sembly language and downloaded into a Parallax PIC 16C54 microcontroller. The first 
program is the time delay program and the second program is the duty cycle modification 
Circuit. 


1. Time Delay Program 


:THIS PROGRAM DELAYS A 30 HZ SQUARE WAVE FROM 1.4 mSEC TO 
;16.3 mSEC (FROM IN PHASE TO INVERTED) 


DEVICE PIC16C54;. XT OSC; WOT OFF,PROTECT OFe 


RESET START 


aes pup ee SD ECLARATIONG 9 e337 PRs Cae ee ee OE EEE ERD 
DELAY EQU RB 
INPUT EQU RA. 2 
OUTPUT EQU RA. 3 
LOOK EQU 08H 
DLY2 EQU O9H 
N1 EQU OAH 
N2 EQU OBH 
N3 EQU OCH 
START MOV IRA, #0100b ;MAKE BIT 2 AN INPUT, BIT 3 AN 
OUTPUT 
MOV tRB,#11111111b ;SETUP REG B AS INPUTS 
MOV LOOK, #2 ; LOOK=2 
WAIT JNB INPUT, WALT ;WAIT TIL POS INPUT TRIGGER 
DINZ LOOK, WAIT :;SAMPLE 2 TIMES TO ENSURE NOT 
GLITCH 
JB RB.7,LDLY 
LCALL PHASE *-PHASE DELAY SUBROUTINE 
LCALL GEN -GENERATE PULSE WIDTH SUBROUTINE 
LUMP WALT ;START OVER 
LDLY LCALL LPHS 


LCALL LGEN 
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LUMP WAIT 


priate riaiiGiG7O-8m SEC PHASE DELAY SUBROUTINE; ;;;3;33s3s35e33 
PHASE MOV DLY2, DELAY ; INPUT DELAY SWITCH 
LOOP3 MOV N1, #16 
LOOPL DINZ N1, LOOP1 ;0 TO 8m SEC DELAY TIL OUTPUT SET 
HIGH 
DINZ DLY2,LOOP3 
RET 
PREETI ee ee ae ee: SEC: DEDAYs 255 oe Roce eRe Be Ss aie ante ae 
LPHS MOV DLY2, DELAY 
CLRB DiLY2 x4 
LOOPS MOV Ni,#17 
LOOP6 DINZ N1,LOOP6 
DINZ DLY2, LOOPS 
RET 
tir acegeca2770-8m SEC PULSE WIDTH SUBROUTINE; :ss333333¢237733732 
GEN SETB RA.3 *SLARL HIGH SIDE DUTY CYCLE ‘1 
MOV N3,#1 2 
LOOP4 MOV N2,#230 ; Z 
LOOP2 DJINZ N2,LOOP2 ;LOOP DELAY FOR 8m SEC 3 
DINZ N3, LOOP4 ; 3 
CLRB RA. 3 :SET OUTPUT LOW 1 
RET 
tect ieecaaGg;8-1l7m SEC PULSE WIDTH SUBROUTINE; ;;iss333¢3333cccc3 
LGEN CLRB RA.3 
MOV N3,#11 
LOOP7 MOV N2,#240 
LOOPS DINZ N2,LOOP8 
DINZ N3, LOOP7 
SETB RA.3 
RET 
ete ee ea evo ee eae a ard en) 8) eee a ee ee ge a ere ah ee ee are tise aaa 


2. Pulse Width Modification Program 


; THIS PROGRAM VARIES THE PULSE WIDTH ON A 30 Hz 
; SQUARE WAVE TO PROVIDE A DUTY CYCLE FROM 5% TO 50% 


DEVICE PIC16C54,XT_OSC,WDT_OFF, PROTECT_OFF 


RESET START 


pricaiaaigG DECLARATIONS; ister rissstssacascebtrcsciie 
DELAY EQU RB 
INPUT EQU RA.2 
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OUTPUT 
LOOK 
DLY2 
Nl 


START 
OUTPUTS 


WAIT 


es 6©« @ 2 *© © © * &© © @ ££ @ 8 8 @ 


fof# # # # ¢ # # 


PULSE 


LOOP3 


LOOP1 


WAIT2 


LOOP 2 
LOOP4 


ror # tf # # F fF 


SETB 
MOV 
MOV 
DINZ 
DINZ 
CLRB 
JB 
SETB 
MOV 
MOV 
DINZ 
DINZ 
CLR 
RET 


RA. 3 
08H 
OSH 


OAH 


(RA, #0100b 


IRE, Fill lit 


INPUT, WALT 
PULSE 
WAIT 


>;0-16 mSEC PULSE SUBROUTINE; 


RA. 3 

DLY2, DELAY 
N1, #8 

N1, LOOP 
DLY2, LOOP3 
RA. 3 
INPUT, WAIT2 
RA. 0 

DLY2, DELAY 
N1, #8 
N1,LOOP4 
DLY2,LOOP2 
RA .0 


;MAKE BITS 2 INPUT, BIT O, l, 
-SETUP REG B AS INPUTS 


-WALT TIL POS INPUT TRIGGER 
:GEN PULSE SUBROUTINE 


ee e« 8s 6 # # # © &@ @& @ @ @ @ 


prop @ f # PF # & F 6 ee RF 


soot envneonwvnevneveeeosvn oe sen avnW@*eeve ee eee ee eel 


Pa ee ee ae ee SS END: 
prrererrerprperevre te Pe eer er eevrer ,prtrrenrervr er erreeo ee? errr erererenreereoereraeere 
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